perf(prover): default the cuda table scheduler to K = num_airs - #911
Merged
Conversation
`table_parallelism()`'s cuda arm scaled K by `available_parallelism()`
(`cores * 2 / 3`). Measured over 881 runs on two RTX 5090 boxes, that is the
wrong shape. All eight core-count curves fit `T(K) = S + max(Tmax, W/K)` within
run-to-run noise, and the work K divides — W ≈ 5.3-8.0 s — is invariant to host
core count over an 8x range, to CPU model, and to rayon pool width: cutting
RAYON_NUM_THREADS 32 -> 4 leaves W alone and merely doubles S, with the best K
still num_airs at every pool width. `available_parallelism()` sizes precisely
that rayon pool, so it is the wrong quantity to scale K by. K is not a thread
count; each table's work runs on the one global pool.
Worst case against the best measured K, over four core counts on both boxes:
cores/3 +30.2 %
cores*2/3 +13.0 % (what this replaces)
constant 12 +7.0 %
num_airs +1.6 % (both non-zero cells inside noise, p = 0.88 / 0.80)
`cores*2/3` fails where it was predicted to: low core counts, K=2 at 4 cores
(+13.0 %) and K=5 at 8 cores (+8.1 %).
Taking the ceiling rather than solving for an optimum is right in both regimes
of the fit: if W/num_airs > Tmax more K strictly helps, and if W/num_airs < Tmax
the extra drivers are floor-limited and cost nothing — the one staging slab is
held 56 % of wall at K=31 and wall time still improves. The old doc comment's
mechanism ("in-flight tables mostly sit in GPU waits") is not what happens —
mean GPU utilisation never exceeded ~38 % at any K — so it is rewritten rather
than re-tuned. What is meant to bound concurrency is memory admission rather
than a count: that is what VramGate is for, and it never binds at the default
budget.
`table_parallelism` now takes `num_airs` and clamps to it, replacing the
`.min(num_airs)` the call site applied. `auto_storage::decide` keeps a bounded
figure through the new `storage_estimate_parallelism()`: `peak_bytes` sums the
transient bytes of the top-k tables, so an unbounded k there sums every table —
measured +27 % at 128 PAGE tables, +44 % at 512 — and would spill proofs to disk
that fit in RAM. Its value is unchanged, so no storage decision moves.
The CPU arm keeps `cores / 3`. The sweep ran only on cuda builds, where the
parallelized work is device-bound; on a CPU-only build every table is pure host
work and none of this evidence transfers.
MauroToscano
added a commit
that referenced
this pull request
Aug 10, 2026
The sweep record moves out of the tree to a gist linked from the PR, so the doc comment can no longer cite scripts/profiling/table-parallelism-sweep/.
MauroToscano
force-pushed
the
perf/table-parallelism-num-airs
branch
from
August 10, 2026 18:09
4049dcb to
67fde58
Compare
The sweep record moves out of the tree to a gist linked from PR #911, and the full defense of the K = num_airs choice (curve fit, rayon-width legs, per-cell p-values) lives there and in the PR body. The code site keeps the conclusion, the mechanism in one sentence, the headline numbers, and the pointer.
Under the cuda feature the unwrap_or_else closure in table_parallelism collapses to a plain num_airs, tripping the lint on the Makefile's cuda clippy pass. Move the cfg split outside the closure: the cuda arm uses unwrap_or, the CPU arm keeps its lazy host_cores() call.
MauroToscano
force-pushed
the
perf/table-parallelism-num-airs
branch
from
August 10, 2026 18:22
67fde58 to
d4c9e05
Compare
Oppen
approved these changes
Aug 10, 2026
ColoCarletti
approved these changes
Aug 18, 2026
GPU Benchmark (ABBA) —
|
MauroToscano
added a commit
that referenced
this pull request
Aug 18, 2026
Brings in the four commits that landed since the campaign base 58160b6: the bump guest allocator default (#869), the VRAM-pressure/R2-race fix (#914), the cuda table scheduler K = num_airs default (#911), and the device-only cliff recovery at R4 DEEP / comp-tree / R3 barycentric (#935). Conflicts resolved (one file, three hunks, all the same collision): - crypto/stark/src/prover.rs — the Stage-2 H-threading parameterized Round1/Round2 over the hasher, while #935 changed the same parameters from shared to mutable so the cliff recovery can download resident device data back into the host buffers. Rule: keep both — main's `&mut` mutability and this branch's `H` parameter. Applied at round_3_evaluate_polynomials_in_out_of_domain_element, round_4_compute_and_run_fri_on_the_deep_composition_polynomial, and compute_deep_composition_poly_evaluations. The recovery paths therefore run through the H-generic signatures; nothing is un-genericized. Everything else merged without conflict. Checked by hand rather than trusted to the textual merge: - crypto/stark/src/gpu_lde.rs — the two sides are disjoint. #935 appends the host-download helpers and the sticky fault hooks; the H-threading edits sit in the tree-building and FRI-commit entries. main's one hunk inside threaded territory is comment-only. - crypto/math-cuda/src/lib.rs — both sides add one `pub mod` to the same alphabetized list (`blake3` here, `faults` on main); both survive. - crypto/math-cuda/src/device.rs — touched only by this branch, so #935's math-cuda edits (barycentric, deep, faults, merkle) do not collide. - prover/tests/calibration.rs and prover/src/auto_storage.rs — #911 splits the scheduler's `k` from the storage estimate's, so both call sites move to `storage_estimate_parallelism()`. This branch never touched either file, so main's version lands whole and the RAM-vs-Disk decision is unmoved. - The `table_parallelism()` call site takes main's `table_parallelism(num_airs)`, which clamps internally to the same range this branch clamped by hand.
MauroToscano
added a commit
that referenced
this pull request
Aug 18, 2026
Carries origin/main (cf3b1e9) onto the flip branch: the bump guest allocator default (#869), the VRAM-pressure/R2-race fix (#914), the cuda table scheduler K = num_airs default (#911), and the device-only cliff recovery at R4 DEEP / comp-tree / R3 barycentric (#935). No conflicts. Both of the resolutions made when main met this campaign's tree were already settled one branch down and arrive whole: - `crypto/stark/src/prover.rs` — Round1/Round2 carry both main's `&mut` and the campaign's `H` parameter, so #935's cliff recovery runs through the H-generic signatures. - `crypto/stark/src/prover.rs` — the device-only main-LDE recovery matches `MainLdeSlot` exhaustively: `Retained` downloads off the resident handle, `Dropped` (RecomputeLde) needs nothing because the fused task rebuilds from the host trace. The flip's own collision surface stayed clear: the renamed alias layer (`DefaultStarkHash`, `DefaultStarkTranscript`) and the `assert_keccak_backend` guard over the cuda fork are untouched by main's gpu_lde edits, and the cuda clippy pass — where this branch resolves to keccak — compiles clean. Gates: stark release 287/0; crypto 72/0 on both round arms; `lfm::` 354 passed / 1 failed / 9 ignored, the same single pre-existing `fibonacci.elf` drift exonerated in RESUME-PA-STAGE6.md §5.7, so zero delta; BLAKE3 host KAT green on both round arms; second-source green; `make lint` clean across all five combos; fmt clean. The cross-version king gate against pre-merge refs stays failing by design — that is the flip's inverted polarity (PA-PLAN §6), not a merge regression.
MauroToscano
added a commit
that referenced
this pull request
Aug 18, 2026
Carries origin/main (cf3b1e9) onto the MMCS integration branch: the bump guest allocator default (#869), the VRAM-pressure/R2-race fix (#914), the cuda table scheduler K = num_airs default (#911), and the device-only cliff recovery at R4 DEEP / comp-tree / R3 barycentric (#935). This is the branch where the two sides genuinely interleave. M-4p2 extracted the round bodies so they take the data they use — `lde_trace`, `composition_parts`, `rap_challenges` — instead of the whole `Round1`/`Round2`, and `multi_prove_batched` reuses those same extractions. #935 works the other way: its recoveries write the resident device data back into those very buffers, which is why upstream widened the round signatures to `&mut Round1`/`&mut Round2`. Neither shape can simply win. Resolution rule, applied to all nine hunks: **keep the extraction, move the mutability onto the extracted parameter.** Each recovery then writes to exactly the buffer its caller owns, and the batched path keeps sharing one implementation with the monolithic one. - `crypto/stark/src/prover.rs` `compute_composition_parts` — `lde_trace` becomes `&mut`; the R2 host-evaluator arm takes #935's recover-then-assert (replacing the old hard abort) against that parameter rather than `round_1_result.lde_trace`. - `crypto/stark/src/prover.rs` `compute_composition_parts` — the `evaluate_dev` arm keeps the extracted `rap_challenges` and the extracted `lde_trace` in the `host_trace_empty` retain flag. - `crypto/stark/src/prover.rs` `round_2_compute_composition_polynomial` — keeps the `CompositionParts` return, and #935's fold of the R2 device parts handle into the session (`set_gpu_composition_parts`) is added after the call, where `round_1_result` is in scope. - `crypto/stark/src/prover.rs` `round_3_evaluate_polynomials_in_out_of_domain_element` — extracted `lde_trace` and `composition_parts` both become `&mut`; the R3 parts OOD arm takes #935's recovery against them. - `crypto/stark/src/prover.rs` `compute_deep_composition_poly_evaluations` — same two parameters become `&mut`; the host DEEP loop's recovery writes through `composition_parts` instead of `round_2_result.lde_composition_poly_evaluations`. - `crypto/stark/src/batched/prover.rs` — the three call sites and `deep_codeword` follow the widened signatures; the FRI combine closure captures `retained_parts` mutably. That closure is `FnOnce` and runs serially, so the capture adds no concurrency requirement. - `crypto/stark/src/prover.rs` — the two `mut` bindings the split moved: the parts the R2 commit recovery writes now live in `round_2_compute_composition_polynomial` (so `computed.parts` is bound `mut` there), and `compute_composition_parts`'s own local is no longer mutated by anything, so it loses the `mut` and the `unused_mut` cfg_attr that went with it. Only the cuda lint pass sees either. Both semantics are live afterwards: nothing is un-genericized, no recovery is dropped, and the parameter each recovery writes to is the one the caller reads next. Gates: stark release 350/0 (RESUME-MMCS-INT.md's 349/0 plus main's new `table_parallelism_stays_within_one_and_num_airs`); debug batched/mmcs 87/0, exactly the recorded baseline; crypto 71+1/0 on both round arms; `make lint` clean across all five combos; fmt clean. `lfm::` reads 345 passed / 19 failed / 9 ignored against a recorded baseline of 349/15/9, and the merge is NOT the cause. Checked out 46798a5 — this branch's own pre-merge tip — and ran the same suite there: 345/19/9, and the 19 failing test names diff byte-identical against the merged tree's. The merge delta is exactly zero; the recorded baseline is stale, drifted by the fixture/toolchain trap already documented in the lfm fixture-drift notes. Independently, every resolution in this merge is inside `#[cfg(feature = "cuda")]` or is a signature mutability change, and that suite runs without cuda, so it could not have moved those tests either way. SEMANTIC-CONFLICT NOTE. The batched path consumes its parts on the host immediately (`parts_builder.absorb`) and never reads the device parts handle, so the recoveries are inert there today — `materialize_composition_parts_host` returns true without touching anything when the evals are already populated, so the widened signatures cost the batched path nothing and cannot trip its asserts. The recovery is only reachable on the monolithic path. Flagged because that is a judgement about reachability, not something a test currently pins.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The cuda arm of
table_parallelism()scales K byavailable_parallelism()(
cores * 2 / 3). Measured over 881 runs on two RTX 5090 boxes, that is the wrongshape — not the wrong constant.
All eight core-count curves fit
T(K) = S + max(Tmax, W/K)within run-to-run noise,and the work K divides (W ≈ 5.3–8.0 s) is invariant to host core count over an 8×
range, to CPU model, and to rayon pool width. The decisive one: cutting
RAYON_NUM_THREADS32 → 4 leaves W alone and merely doubles S, with the best K stillnum_airsat every pool width.available_parallelism()sizes precisely that rayonpool, so it is the wrong quantity to scale K by — K is not a thread count, each
table's work runs on the one global pool.
Worst case against the best measured K, over four core counts on both boxes:
cores/3cores*2/3(what this replaces)num_airscores*2/3fails where predicted: low core counts, K=2 at 4 cores (+13.0%), K=5 at 8cores (+8.1%). No K below
num_airsis significantly faster thannum_airsanywhereon either box (every such cell p ≥ 0.38 at n=8).
Taking the ceiling rather than solving for an optimum is right in both regimes of the
fit: if
W/num_airs > Tmaxmore K strictly helps; ifW/num_airs < Tmaxthe extradrivers are floor-limited and cost nothing — the one staging slab is held 56% of wall
at K=31 and wall time still improves.
The old doc comment's mechanism ("in-flight tables mostly sit in GPU waits") is not
what happens — mean GPU utilisation never exceeded ~38% at any K — so it is rewritten
rather than re-tuned. What is meant to bound concurrency is memory admission rather
than a count: that is what
VramGateis for.auto_storageis held boundedtable_parallelismnow takesnum_airsand clamps to it, replacing the.min(num_airs)the call site applied.
auto_storage::decidekeeps a bounded figure through the newstorage_estimate_parallelism():peak_bytessums the transient bytes of the top-ktables, so an unbounded k there sums every table — measured +27% at 128 PAGE tables,
+44% at 512 — and would spill proofs to disk that fit in RAM. Its value is unchanged,
so no storage decision moves. Two tests pin both halves.
The CPU arm is untouched
cores / 3stays. The sweep ran only on cuda builds, where the parallelized work isdevice-bound; on a CPU-only build every table is pure host work and none of this
evidence transfers.
Evidence
The full sweep record — both rounds' write-ups, every result CSV and the re-runnable
harnesses, so every number above is checkable — lives in a gist to keep this diff
reviewable: https://gist.github.com/MauroToscano/54cdbf03da9a69517ae30c964efc2827
(filenames are the original
scripts/profiling/table-parallelism-sweep/pathsflattened with
__;00_INDEX.mdmaps the layout).Honest limits
epoch-size-log2 21, ~31 tables. Production tables are much larger, so memory per
concurrently-admitted table is larger and the
auto_storageinteraction is unverifiedat that scale — a big-block check is a merge gate, not something this PR establishes.
taskset -c 0-7is oneCCD with 32 MB L3 vs the 3D part's much larger cache).
T(1) − T(best), not instrumented per phase.S + W/Kwith nosaturating resource, so a tolerance-knee is derived and reads 8 on one box and 16 on
the other at the same 5% tolerance.
Verified
cargo fmt --checkclean;cargo clippy --all-targets -p stark -p lambda-vm-proverclean.
auto_storage_tests8 passed (needs--features disk-spill);table_parallelism_stays_within_one_and_num_airspasses. The cuda-gated test and thecuda arm itself cannot compile here — no local CUDA toolchain — so both were verified by
reading plus a non-cuda build.